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SUMMARY 


Objective 

The  purpose  was  to  develop  and  test  an  objective  procedure  to  determine  the  relative  difficulty  of  Air  Force 
jobs.  Also  investigated  were  (a)  the  measurement  of  task  difficulty  to  allow  comparability  across  specialties,  (b)  the 
quantitative  appraisal  of  job  demands  based  on  component  tasks  being  performed,  and  (c)  the  comparability  of  job 
difficulty  to  job  aptitude  requirement. 

Background 

The  present  work  is  the  culmination  of  a  long  stream  of  research  and  development  examining  methodologies 
for  systematically  determining  relative  aptitude  requirements  of  Air  Force  jobs.  Such  methodologies  are  needed 
since  there  are  no  empirically  based  procedures  for  establishing,  adjusting,  or  verifying  the  aptitude  cutoff  score 
requirements  published  in  Air  Force  Regulations. 

Early  research  in  this  area  offered  substantial  support  for  the  use  of  I  ime- to- learn  as  a  key  element  in 
measuring  the  ability  requirements  of  Air  Force  jobs.  In  addition,  the  level  of  aptitude  required  for  successful 
performance  of  a  task  was  found  to  be  conceptually  inseparable  from  the  time  required  to  learn  to  perform  the  task 
at  a  satisfactory  level.  Thus,  a  benchmark  scaling  technique,  in  which  anchor  tasks  are  used  to  describe  each  level 
on  the  scale,  was  developed  to  measure  relative  difficulty  from  which  relative  aptitude  requirements  could  be 
inferred.  These  results  may  be  used  by  Air  Force  managers  to  establish  entry-level  aptitude  requirements  and  to 
assign  individuals  to  career  specialities  more  accurately. 


Approach 

The  study  was  based  on  task-level  specifications  of  learning  difficulty.  The  specifications  were  provided  by 
two  complementary  sources  of  expert  ratings.  One  source  included  occupational  survey  data,  that  is  routinely 
collected  on  most  Air  Force  jobs.  Such  data  contain  relative  ratings  of  task  difficulty  collected  from  knowledgeable 
supervisory  personnel  within  each  specialty.  Secondly,  contract  job  analysts  provided  benchmark  ratings  of 
selected  tasks  across  specialties.  Collection  of  benchmark  data  permitted  the  development  of  techniques  for 
calibrating  the  supervisors'  ratings  to  a  standard  reference  base  such  that  tasks  in  one  specialty  could  be  compared 
to  tasks  in  other  specialties.  Data  on  the  relative  lime  spent  by  job  incumbents  on  each  task  also  were  available  in 
the  occupational  survey  data.  These  data  were  used  to  weight  the  relative  difficulty  of  each  task  when  computing 
aggregate  estimates  of  learning  difficulty  for  each  enlisted  Specialty- 


Specifics 

The  Comprehensive  Occupational  Data  Analysis  Programs  (CODAP)  package  was  used  for  the  analysis  of  task 
level  data.  Interraler  reliability  and  correlation  techniques  were  used  to  assess  I  he  agreement  among  supervisors 
and  job  analysts  in  the  ratings  of  task  difficulty.  Regression  equations  were  used  to  calibrate  relative  ratings  on  the 
benchmark  scale.  The  calibrated  ratings  then  were  combined  with  average  lime-spent  data  to  determine  the 
relative  difficulty  of  individual  jobs  and  specialty  groups.  The  resultant  values  were  designated  ATDPI'TS 
(average  task  difficulty  per  unit  time  spent). 

Both  supervisory  ratings  and  the  contract  job  analyst  ratings  proved  to  be  highly  reliable.  In  addition,  a  high 
degree  of  relationship  was  shown  between  the  supervisory  ratings  and  the  contract  job  analyst  ratings.  The 
benchmark  scales  provided  a  highly  reliable  means  of  obtaining  task  diffculty  ratings  that  were  comparable  across 
specialties. 


Conclusions/Recommendations 

The  methodology  developed  and  implemented  can  be  applied  objectively  to  evaluate  the  relative  aptitude 
requirements  of  Air  Force  jobs.  Air  Force  managers  now  have  systematic  and  empirical  data  with  which  to  order 
jobs  relative  to  each  other  based  on  the  level  of  talent  required.  It  is  recommended  that  this  methodology  be 
considered  for  use  in  operational  realignment  of  current  aptitude  requirements. 
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PREFACE 


The  purpose  of  t his  research  effort  was  to  develop  and  apply  a  methodology  for  the  evaluation  of 
aptitude  requirements  for  Air  Force  enlisted  specialties  based  on  task  difficulty.  This  effort  was  the  initial 
phase  of  a  project  in  response  to  RPR  73-17.  Minimum  Aptitude  Requirements  for  Airmen  AFSCs.  to  derive 
empirically-based  minimum  aptitude  requirements  for  Air  Force  enlisted  specialties.  The  research  is  in 
support  of  the  Force  Acquisition  and  Distribution  System  subthrust,  and  Manpower  and  Force  Management 
thrust. 

Dr.  Raymond  E.  Christal  of  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  deserves  specific 
credit  for  the  evolution  of  this  research  from  an  original  concept  for  determining  aptitude  requirements  to 
the  development  of  a  complex  methodology  which  allows  assessment  of  the  learning  difficulty  of  each 
individual  job  in  the  Air  Force.  He  is  due  special  recognition  for  working  very  closely  and  conscientiously 
with  the  authors  throughout  the  period  of  this  report.  Acknowledgement  is  also  due  Mr.  Fred  Hart.  Rinton. 
Inc.,  Alexandria,  VA,  for  leading  a  very  large  and  complex  data  collection  effort  and  to  Mrs  Nancy  Perrigoof 
AFHRL  for  laying  the  early  groundwork  for  the  project.  Special  appreciation  goes  to  the  Air  Force  Manpower 
and  Personnel  Center  Directorate  of  Personnel  Resources  and  Distribution  (AFMPC/MPCR).  Utilization 
Policy  and  Control  Division  (AFMPC/MPCRP),  and  USAF  Classification  Branch  (AFMPC/MPCRPCP)  for 
their  long-standing  support  of  this  project  under  RPR  73-17.  Finally,  this  project  could  not  have  been 
successfully  completed  without  the  expert  programming  and  seemingly  continuous  consultation  of  Mr. 
Johnny  Weissmuller  of  AFHRL, 
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APTITUDE  REQUIREMENTS  BASED  ON  TASK  DIFFICULTY: 

METHODOLOGY  FOR  EVALUATION 

I.  INTRODUCTION 

Eligibility  for  entry  into  the  various  Air  Force  career  ladders  is  based  primarily  on  the  minimum  aptitude 
'(■ore  cutoff  on  one  or  more  of  the  composites  of  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  (AFR 
PM.  1 077).  There  are  four  ASVAB  composites  in  use  by  the  Air  Force:  Mechanical.  Administrative.  General, 
Electronics.  An  individual  s  percentile  score  on  these  composites  is  the  principal  factor  for  determining  eligibility 
for  entry  level  jobs.  Although  this  report  is  primarily  a  description  of  methodology  and  procedure  for  the 
evaluation  of  aptitude  requirements,  the  essential  problem  being  examined  is  the  validity  of  the  relative  ordering 
of  assigned  ASVAB  minimums  in  comparison  with  the  computed  relative  order  of  difficulty  of  the  jobs  based  on 
work  performed. 

The  correlation  of  success  in  training  with  aptitude  composite  scores  and  the  technical  school  pass/fail  rates 
are  the  primary  data  used  by  the  Air  Force  to  set  aptitude  minimums.  Relative  correspondence  between  success  in 
training  and  each  of  the  aptitude  composites  is  used  to  establish  the  aptitude  area  (M,  A.  G,  or  E)  for  a  specific 
specialty,  and  the  pass/faii  rate  is  used  to  adjust  the  minimum  cutoff  score  (Maginnis.  Uchima.  &  Smith,  1975a. 
1975b,  1975c).  Although  this  appears  to  be  a  valid  and  empirically  based  decision  logic,  there  exist  some  deep- 
seated  problems.  The  standards  for  successful  completion  of  courses  appear  to  be  arbitrarily  set  and  tend  to 
fluctuate  with  the  number  of  trainees  needed.  This  problem  is  further  compounded  by  a  training  time  and  aptitude 
trade-off.  That  is,  an  unsuccessful  trainee,  rather  than  being  washed-out,  may  be  recycled  through  the  same  course 
until  a  passing  score  is  achieved.  Thus,  a  potential  failure  has  been  converted  to  a  successful  completion  by 
allowing  more  time  to  learn.  Christal  (1976)  presents  a  detailed  description  of  the  problems  in  the  prediction  of 
training  success  from  aptitude  test  scores. 

The  consequences  of  setting  appropriate  aptitude  levels  for  entry  into  Air  Force  specialties  (AFSs)  go  beyond 
the  immediate  impact  on  training  outcomes.  For  example,  lowering  a  requirement  from  the  80th  to  the  60th 
percentile  could  double  the  number  of  eligible  volunteers  for  a  particular  occupation  (Christal.  1974). 
Inappropriate  assignment  of  aptitude  requirements  can  have  a  significant  impact  on  job  attitudes —  individuals 
assigned  to  jobs  that  do  not  fully  utilize  their  talents  tend  to  experience  boredom:  individuals  assigned  to  positions 
requiring  more  talent  than  they  have  tend  to  experience  a  sense  of  frustration  (Locke.  1976).  Both  circumstances 
can  adversely  affect  absenteeism,  retention,  and  learning  rate  (Brayfield  &  Crockett.  1955:  Taylor  &  Weiss.  1972; 
Waters  &  Roach.  1971,  1973;  Wyatt.  Langdon,  &  Stock.  1937).  The  data  collected  in  this  study  go  beyond  the 
training  school  setting  and  reflect  the  actual  difficulty  of  a  given  job  in  the  operational  setting. 

The  overall  objective  of  the  present  effort  was  to  design,  develop,  and  test  a  methodology  that  could  be  applied 
effectively  and  objectively  to  determine  the  relative  difficulty  of  Air  Force  jobs.  The  two  major  sub-objectives  were 
to  develop  procedures  for  (a)  the  measurement  of  task  difficulty  such  that  tasks  would  be  comparable  across 
specialties  and  (b)  the  quantitative  appraisal  of  job  demands  based  on  component  tasks  being  performed. 


II.  APPROACH 

Conceptual  Framework 

Empirical  data  are  not  necessary  to  realize  that  there  is  tremendous  variance  both  in  job  demand  levels  and  in 
individual  learning  rates.  It  is  not  difficult  to  imagine  some  AFSs  in  which  those  airmen  with  the  lowest  aptitude 
(the  slowest  learners)  can  perform  very  successfully  after  only  a  short  training  period.  On  the  other  hand,  there  are 
also  AFSs  in  which  the  airmen  with  the  highest  aptitude  (the  fastest  learners)  must  undergo  extensive  on-the-job 
training  even  after  long  periods  (30  or  more  weeks)  of  formal  training.  The  need  to  determine  the  relationship 
between  aptitude  and  learning  time  has  become  more  acute  as  has  the  necessity  of  defending  empirically  the 
aptitude  levels  that  are  set  as  occupation  entry  requirements. 

Several  educational  researchers  offer  support  for  the  use  of  time-lo-learn  as  a  key  element  in  measuring  the 
ability  requirements  of  Air  Force  jobs.  Aptitude  can  be  looked  at  as  something  that  remits  in  an  individual  being 
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ready  to  learn  "rapidly**  in  a  specific  situation  (Lronbach  X  Snow,  1977).  Furthermore,  (iroiibach  ami  Snow  rlaiin 
that  students  will  likely  differ  in  the  time  they  require  to  learn,  given  the  same  material  and  inslruetional 
procedures.  Recent  documentation  by  (Hettinger  and  Vthite  (1979)  offers  additional  evidence  in  Mipporl  of  tiine-lo* 
learn  as  a  predictor  of  achievement  and  aptitude.  These  authors  indicate  that  the  time-to-lcarn  concept  makes  no 
assumptions  about  lire  intelligence  required  to  perform  a  task,  but  deals  only  with  performance  under  natural 
conditions.  This  literature  in  addition  to  earlier  work  by  Carroll  (in  Block  K  \nderson.  197.7:  in  (iron  bach  <Jt  Snow. 
1977;  and  in  Rruinboltz.  1965)  provides  strong  support  for  the  integration  of  the  time-to-learu  concept  into  the  Air 
Force  classification  and  assignment  systems. 

The  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  been  conducting  research  into  this  problem  for 
several  years.  The  methodology  discussed  in  this  report  has  greatly  benefited  and  evolved  from  previous  work 
conducted  by  Christal  (1971)  and  Fugill  (1971.  1972a.  1972b.  1975)  in  developing  the  Air  Force  job  inventory 
methodology  and  investigating  the  area  of  task  difficulty  and  benchmark  scale  use.  The  approach  was  based  on 
task  level  specifications  of  learning  difficulty  provided  by  two  complementary  sources  of  expert  ratings:  (a) 
knowledgeable  supervisory  personnel  within  each  specialty  provided  relative  ratings  of  task  difficulty,  and  (b) 
contract  job  analysts  provided  benchmark  ratings  of  selected  tasks  across  specialties.  Access  to  the  benchmark 
ratings  permitted  the  development  of  techniques  for  calibrating  the  relative  ratings  to  a  standard  reference  base 
and  for  generating  aggregate  estimates  of  learning  difficulty  for  every  enlisted  specialty  in  the  Air  Force. 


Task  Difficulty 

The  concept  of  task  difficulty  was  operationally  defined  in  terms  of  the  time  it  lakes  to  learn  to  do  a  task 
satisfactorily.  Fugill  (1971)  demonstrated  that  in  spite  of  the  complexity  of  the  concept,  highly  reliable  ratings  of 
relative  task  difficulty,  as  defined  above,  could  be  obtained  from  supervisory  job  incumbents  from  a  given  career 
field.  FugilFs  (1972b)  research  consistently  demonstrated  a  high  relationship  (r  =  .89)  between  time-to-learn  (task 
difficulty)  and  task  aptitude,  "the  level  of  aptitude  required  to  insure  satisfactory  performance  of  a  given  task*'  (p. 
1).  The  aptitude  requirements  research  documented  in  this  report  has  proceeded  on  the  basis  that  the  aptitude 
level  required  to  learn  a  job  can  be  inferred  from  a  measurement  of  the  average  difficulty  of  that  job.  This 
assumption  is  primarily  based  on  FugilFs  (1972b)  conclusion  that  relative  task  aptitude  is  conceptually  inseparable 
from  relative  task  difficulty  when  difficulty  is  measured  in  terms  of  tile  time  needed  to  learn  to  perform  a  task 
satisfactorily. 


Occupational  Survey  Data  Base 

The  basic  data  used  in  the  identification  of  tasks  for  the  estimation  of  task/job  difficulty  indices  came  from 
the  occupational  survey  data  routinely  collected  by  the  USAF  Occupational  Measurement  Center.  Briefly,  the  job 
inventories  used  in  the  periodic  occupational  surveys  of  Air  Force  jobs  are  developed  by  creating  a  duly  outline 
and  a  listing  of  task  statements  based  on  job  descriptions,  course  training  standards,  and  other  published  materials 
(Christal.  1974).  Tasks  are  then  organized  within  duty  categories  and  the  task  list  revised  based  on  work-site 
observation  of  the  job  and  input  from  technical  specialists.  When  finalized,  the  job  inventory  is  administered  to  job 
incumbents  within  the  specialty  to  collect  information  about  the  relative  amount  of  work-time  spent  on  the  tasks 
which  they  perform,  using  a  1-9  point  scale  ranging  from  fckA  Very  Small  Amount**  to  "A  Very  Large  Amount." 
These  data  are  compiled  in  a  computer-generated  job  description  to  provide,  among  other  information,  an 
estimation  of  the  percentage  of  incumbents  who  perform  each  task  and  the  average  percentage  of  time  spent  on 
each  task  by  those  in  the  specialty  who  perform  it.  This  same  information  can  be  reported  for  any  group  of 
individuals  who  can  be  defined  by  available  background  variables  such  as  time  in  service,  grade,  education,  and 
time  in  job. 

The  same  duty/task  list  is  administered  to  supervisors  who  are  asked  to  rate  the  tasks  on  task  difficulty,  based 
on  how  much  time  is  required  to  learn  the  task,  using  a  1-9  point  scale  ranging  from  "A  Very  Small  Amount"  to  "A 
Very  Large  Amount."  These  ratings  are  compiled  to  give  an  estimate  of  the  task  difficulty  of  each  task  compared 
with  other  tasks  in  the  inventory. 
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Analytic  Techniques 

The  (  comprehensive  Occupational  Data  Analysis  Programs  (UlllAI1)  package  developed  by  AFHRL  (Christa!. 
1071:  Morsb.  Madden.  &  Christal.  1961)  was  the  data  ana)\tie  tool  used  for  this  research.  The  CODAP  system  was 
ideally  suited  lor  this  type  of  analysis.  Computer  analysis  of  all  rating  data  began  with  the  measurement  of  the 
degree  of  inlerrater  agreement  among  all  raters,  computed  using  the  intra-class  correlation  coefficient  (Rjj) 
described  by  Haggard  (1058)  and  Lindquist  (105.1).  This  reliability  coefficient  is  a  measure  of  the  interclass 
correlation  among  raters.  As  discussed  in  Guilford  and  Fruchter  (1071).  each  coefficient  ( R  j  j ) .  taken  to  be  an 
indication  of  the  reliability  of  a  single  rater's  ratings,  can  he  used  to  infer  the  reliability  of  a  group  of  raters  (R^) 
(p.  264).  B>  averaging  each  set  of  ratings  across  the  number  of  raters  rating  each  ta.-k.  group  reliability  coefficients 
(Rj.  j.)  for  all  measures  can  be  computed. ^  The  interrater  reliability  coefficient  as  applied  to  task  factor  ratings  is 
described  by  Good\  (1076)  and  Thomson  and  Goody  (1070).  (u  addition,  correlation/regression  techniques,  the 
calculation  of  average  task  ratings  across  raters,  and  the  generation  of  adjusted  task  difficulty  values  based  on  I  he 
benchmark  equations  were  used  in  the  specific  analyses  for  task  ratings.  The  analytic  techniques  are  further 
discussed  in  the  description  of  procedures  to  develop  task  and  job  difficulty  indices. 


III.  DETERMINATION  OF  TASK  DIFFICULTY 
Development  of  Benchmark  Scales 

Ratings  of  task  difficulty  within  specialties,  as  routinely  obtained  in  conjunction  with  occupational  surveys, 
are  useful  in  comparing  the  relative  difficulty  for  tasks  and  jobs  within  career  ladder>.  However,  a  method  was 
needed  for  comparing  difficulty  and  aptitude  levels  for  tasks  across  career  ladders. 

The  use  of  benchmark  scales  provides  very  reliable  ratings  of  task  difficulty  which  allow  for  comparisons  of 
the  relative  difficulty  of  tasks  not  only  within  a  given  specialty  but  also  across  any  number  of  specialties  measured 
by  the  same  benchmark  scale.  The  benchmark  scale  is  used  as  a  standard  reference  for  calibrating  ratings  obtained 
within  specialties  so  as  to  he  comparable  across  all  specialties  in  an  aptitude  area.  The  feasibility  of  using 
benchmark  scales  to  measure  task  difficulty  was  demonstrated  b\  Fugill  (1971.  1972a.  1972b)  and  further 
discussed  by  Fugill  (1671)  and  Christal  (1974).  Peters  and  McCormick  (1966).  in  a  comparative  study,  obtained 
results  which  demonstrated  that  task-anchored  (benchmark)  scales  resulted  in  more  reliable  ratings  of  several  job 
factors  than  did  numerically  anchored  scales. 

Considerable  thought  was  given  to  the  number  of  points  to  be  employed  on  the  benchmark  scale.  Lissitz  and 
Green  (1975)  briefly  reviewed  the  literature  in  this  area  and  found  no  conclusive  evidence  to  support  anv  specific 
number  of  rating  points.  Research  on  time-spent  scales  by  Carpenter.  Giorgia.  and  McFarland  (1975)  suggests  that 
there  is  little  difference  in  reliability  but  a  potential  increase  in  validity  with  an  increase  in  the  number  of  rating 
options  f rofii  7  to  9  to  25  and  even  to  100  points.  These  results  in  conjunction  with  research  by  Christal  and  Madden 
(1960)  and  Madden  (1960.  1961)  on  the  importance  of  familiarity  in  evaluative  judgments  in  job  evaluation 
directed  this  research  to  a  25-point  benchmark  scale  on  which  the  rater  would  be  carefully  trained,  on  both  the 
tasks  anchoring  the  scale  and  the  tasks  to  be  rated,  prior  to  applying  the  scale. 


Electronics,  Mechanical,  and  General/ Administrative  Benchmark  Scales 

Task  difficulty  benchmark  scales  were  developed  separately  for  the  Electronics.  Mechanical,  and  General/ 
Administrative  aptitude  areas  as  differentiated  by  the  ASVAB.  For  a  given  aptitude  area,  a  set  of  15  specialties  was 
selected  which  best  represented  aptitude  area  complexity  and  provided  a  variety  of  tasks  from  which  benchmark 
tasks  could  he  selected.  Ail  specialties  used  in  the  development  of  the  benchmark  scales  are  shown  in  Appendix  A. 

Table  l  provides  a  summary  of  interrater  reliability  statistics  for  the  relative  difficulty  ratings  collected  from 
specialties  used  in  the  benchmark  scale  development.  Using  a  distribution  of  these  ratings  and  the  criteria  outlined 
in  Table  2.  40  tasks  were  selected  from  each  specialty  to  develop  a  set  of  600  benchmark  tasks  in  the  Mechanical 


*Ciiilford  and  Ft  ter  {197».  p  *  explain  how  ihe  fan  ***  eom puled  from  an  Rjj  and  k  ra'ers. 
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aiui  Electronics  aptitude  areas.  For  the  general  benchmark  st  ale.  (d)  tasks  were  selected  from  eac  h  of  the  15 
specialties  to  produce  a  900-task  list.  For  purposes  of  discussion,  only  tin*  oOO-task  list*  w  ill  in*  referenced  although 
essential!)  the  saint*  procedures  were  followed  witli  the  900  tasks  in  (In*  General  heiichmark  pool.  (See  Appendix  A 
for  complete  interrater  reliability  statistics.) 


Table  I.  Summary  of  Within-Specialty  Inlerrater  Reliability  (R^) 
Indices  for  Specialties  used  in  Development  Phase 


Aptitude 

Area 

Range  of 

Median 

Kkk 

\ 

AFS 

Mean  Number 
of  Raters 

General/ Administrative 

.94  -  .98 

.900 

15 

63. 1 

Mechanical 

be 

oc 

.9 12 

15 

68.9 

Electronics 

.93  -  .99 

.955 

15 

01.7 

Note.  For  all  within-i-periall  v 

rating*,  the  average  number 

of  ruler-  per  l« 

a>k  (k)  ranged  from  20  lo  )IHI. 

Table  2.  Benchmark  Task  Selection  Criteria 


1.  Eliminate  supervisory  tasks 

2.  Capture  range  of  difficulty 

3.  Select  on  high  rater  agreement  (Low  Sl>) 
f.  Select  tasks  performed  by  first-termers 
5.  Select  well  known  tasks 

0.  Select  easily  observed  tasks 
7.  Select  on  high  face  validity 


A  panel  of  8  to  14  job  analysts  was  convened  for  each  aptitude  area.  The  panels,  which  consisted  of  contract 
personnel  considered  expert  in  the  aptitude  area,  obtained  detailed  task  level  information  from  technical  school 
instructors  arid  job  incumbents,  and  observed  task  performance  at  approximately  10  operational  locations  for  each 
aptitude  area.  After  gaining  familiarity  witli  each  task  in  the  list,  each  panel  member  provided  an  independent 
rank-ordering  of  the  (>00  tasks,  placing  the  task  which  required  the  least  learning  time  at  number  I  and  the  task 
requiring  the  greatest  learning  time  at  number  600.  The  final  rankings  represent  the  relative  ordering  of  the  600 
tasks  on  the  dimension  of  learning  time,  without  regard  to  AFS.  Interrater  reliability  estimates  for  the  rank 
ordering  among  judges  for  each  aptitude  area  are  given  in  Table  3.  In  all.  for  the  three  aptitude  areas.  2.100  tasks 
were  independently  rank-ordered  by  a  team  of  8  to  14  raters,  resulting  in  approximately  21,000  rank-order 
estimates. 


Table  2.  Interrater  Reliability  (R^)  for  Raf,k  Ordering  of 
Aptitude  Area  Benchmark  Tasks 


Aptitude 

Area 

Rkk 

N 

Tasks 

N 

AFS 

N 

Rater**/ 

Speriallv 

General/ Administrative 

.90 

<HH) 

15 

12 

Mechanical 

.97 

000 

15 

8 

Electronics 

.90 

000 

15 

8 

Note.  For  all  rank  ordering.  I  In*  average  number  of  raters*  |>er  ia-k  (k)  wa>  equal  lo  N  rater>. 
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I'hr  ranking  proredure  ti>rd  was  »m*  in  which  l In*  judge*  made  pair- wise  comparisons  of  tasks  on  which  thev 
wcrt*  considered  expert.  'This  procedure  resulted  in  a  rank-ordered  list  of  la>k*  which,  it  was  fell,  more  aeenrateK 
captured  the  variance  of  the  difficulty  ol  the  tasks  tlian  would  a  9-point  rating  system.  However,  the  resulting 
distribution  was  understandably  rectangular  in  shape  and  thus  did  not  lend  itself  to  the  development  of  a 
benchmark  scale  with  equal  intervals.  I  hi*  solution  to  this  problem  was  based  on  the  collection  of  9-point 
supervisory  ratings  ol  the  600  selected  tasks.  I  best*  relative  ratings  were  collected  from  approximately  .*>0 
supervisors  from  each  of  the  15  specialties  who  rated  every  task  in  the  list,  not  just  those  selected  from  their 
specially.  The  resulting  distribution  from  these  ratings  approximated  a  normal  curve.  An  equal  percentile 
conversion  program  in  llieGODAP  package  was  used  to  convert  the  task  distribution  preserving  the  order  from  the 
ranking  procedure  into  tin*  normal  distribution  obtained  from  the  rating  procedure.  This  converted  distribution 
was  used  to  develop  a  close  approximation  of  an  equal-interval  benchmark  scale. 

Based  on  the  pane!  rankings  and  tin*  supervisory  ratings  of  the  600  benchmark  tasks,  two  tasks  were  selected 
to  represent  each  ol  tin*  learning  difficulty  levels  of  a  25-point  scale.  The  distribution  of  the  mean  ranks  of  I  lie  600 
tasks  was  divided  into  2;>  equal  intervals.  1  asks  were  selected  w  hich  were  close  to  each  interval  midpoint  v  alue  and 
tor  which  the  standard  deviations  of  both  the  witliimspecialty  ratings  and  the  contractor  rankings  were  relatively 
low.  indicating  that  both  sets  of  judges  agreed  on  the  difficulty  level.  Tasks  were  chosen  w  hich  were  widely  known 
or  frequently  performed,  and  not  unique  to  a  single  specialty.  The  final  criterion,  face  validity,  was  especially 
important  in  the  task  selection  process  inasmuch  as  these  tasks  were  to  he  used  as  examples  that  would  anchor  the 
various  points  on  the  benchmark  scales.  (See  Appendix  B  for  complete  benchmark  scales.) 

Procedural  Guides 

Accurate  application  of  the  benchmark  scales  requires  detailed  knowledge  of  the  benchmark  tasks  as  well  as 
tile  tasks  being  rated.  A  procedural  guide  was  developed  for  each  scale  describing  the  benchmark  tasks.  This  guide 
was  developed  for  tin*  use  of  the  panel  of  expert  raters  who  would  applv  the  scales. 

There  are  two  parts  to  tin*  procedural  guides:  Part  I  introduces  each  panel  member  to  the  task  of  assessing 
learning  difficulty  and  rating  tin*  tasks;  Part  II  presents  the  25-point  scale  and  provides  a  one-page  description  of 
each  of  the  50  tasks  on  the  scale.  This  description  includes  the  staled  task  difficulty  level,  the  task  title,  the 
specialty  from  which  it  was  selected,  a  narrative  description  of  any  specific  equipment  associated  with  the  task,  a 
narrative  describing  the  task  performance,  and  an  explanation  of  the  skills  and  knowledge  required  to  learn  the 
task.  Examples  from  the  Mechanical  Procedural  Guide  art*  included  in  Appendix  C. 

Task  Bating  Using  the  Benchmark  Scales 

The  benchmark  scales  and  procedural  guides  were  developed  to  provide  task  ratings  which  were  comparable 
both  within  and  across  specialties  within  an  aptitude  area.  In  order  to  obtain  such  information,  it  was  necessary  to 
apply  the  same  benchmark  scale  to  all  specialties  in  an  aptitude  area.  This  was  accomplished  by  comparing  a 
carefully  selected  subset  of  tasks  from  each  specialty  to  he  assessed  with  the  tasks  on  the  appropriate  benchmark 
scale  and  assigning  the  respective  rating  to  each  task  in  the  subset.  Regression  techniques  were  then  used  to 
estimate  the  difficulty  of  the  remaining  tasks  in  the  job  inventory  from  the  data  available  from  the  subset  of  tasks. 

Using  criteria  similar  to  those  used  in  the  selection  of  the  benchmark  sets  (Table  2).  60  tasks  were  selected 
from  each  remaining  specialty  in  the  aptitude  area  for  evaluation  by  the  contract  job  analysts  using  the  benchmark 
scales.  Specialties  used  in  the  application  of  the  benchmark  scales  are  indicated  in  Appendix  A.  In  the  application 
phase.  102  specialties  were  evaluated,  approximately  34  technical  school  and  64  operational  site  visits  were  made, 
and  approximately  6,100  tasks  were  rated  by  12  to  14  raters,  resulting  in  over  79.000  ratings.  Again,  each  task 
selected  was  studied  in  depth  at  the  appropriate  technical  school,  as  well  as  at  two  or  more  operational  work  sites, 
by  a  panel  of  aptitude  area  experts.  Panels  consisted  of  12  to  14  members,  with  two  independent  teams  of  six  or 
seven  analysts  observing  the  same  tasks  at  separate  locations.  After  accumulating  considerable  information  about 
each  task,  the  panel  members  independently  provided  benchmark  difficulty  ratings  on  the  60  selected  tasks  from 
each  specialty  using  the  appropriate  benchmark  scale.  Interrater  reliability  statistics  for  these  ratings  are 


summarized  in  Table  4.  Complete  rater  reliability  statistics  on  the  tasks  for  all  speeiallies  studied  are  given  in 
Appendix  A. 


Tahir  t.  Summary  of  Benehniark  Hating  Interrater  Reliability  (R^) 
Indices  for  Specialties  l  sed  in  Application  Phase 


Aptitude* 

Area 

Nailer  of 

Kkk 

Median 

«kk 

\ 

AFS 

Mean  Number 
of  Raters 

General/ Administrative 

.87  -  .68 

.6.7 

77 

1  1.0 

Mechanical 

.88  -  .68 

.67 

27 

18.2 

Electronics 

.62  -  .68 

.67 

22 

12.7 

Calibration  of  Supervisory  Estimates  to  the  Benchmark  Scale 

The  benchmark  ratings  of  the  sample  of  (>0  tasks  within  each  specialty  were  used  to  estimate  the  relative  ia>k 
difficulty  of  all  tasks  in  a  specialty  using  standard  regression  analysis.  The  use  of  the  benchmark  scales  allow?-  a 
task  difficulty  value  to  be  estimated  for  even  task  in  the  inventory  for  the  AFS  under  consideration.  This  value,  in 
turn,  provides  tin*  means  by  which  tasks  and  individual  jobs  can  be  compared  not  onl\  in  relation  to  other  task? 
and  jobs  within  tin*  same  specialty,  but  also  relative  to  tasks  in  other  specialties  within  tin*  same  aptitude  area.  V 
separate  regression  equation  wa>  used  lor  each  APS.  a>  the  relationship  between  the  expert  ratings  and  relative 
ratings  was  unique  for  each  specially. 

The  benchmark  difficult)  ratings  and  the  supervisor)  difficult  ratings  of  the  same  60  tasks  were  input  as  the 
dependent  and  independent  variables,  respective!),  in  a  two-variable  linear  regression  problem  for  each  specialty 
The  equation  took  the  following  form: 

(V  =  a  +  hx) 


t 


where:  V*  is  adjusted  task  difficult) 
a  is  a  constant 
b  is  a  regression  coefficient 

x  is  a  composite  supervisory  rating  of  relative  task  difficult 

The  resulting  equations  were  then  applied  to  the  supervisory  ratings  of  all  tasks  in  the  speeiallies  and  an  adjusted 
difficulty  rating  was  estimated  for  each  task.  In  all.  adjusted  difficulty  ratings  were  estimated  for  approximated 
77.000  tasks. 

Summary  Results  of  Task  Difficulty  Assessment 

The  reliability  and  validity  of  the  data  gathered  in  this  effort  were  investigated  to  insure  I  hat  overall 
methodology  was  sound.  Single  rater  reliability  coefficients  (Rp)  for  all  measures  ranged  from  .16  to  .71.  Group 
reliability  coefficients  (R^)  for  all  measures  ranged  from  .86  to  .68.  (See  Appendix  A  for  complete  rcliahilit) 
statistics.)  Preliminary  investigation  has  shown  that  the  range  of  rcliahilit)  estimates  is  larged  determined  b\  the 
high  variability  of  task  learning  difficulty  across  differences  in  aircraft,  equipment,  or  commands.  Additional 
research  is  currently  being  conducted  to  determine  the  reasons  for  instances  of  low  interrater  agreement. 

As  shown  in  Table  7.  correlations  between  the  benchmark  ratings  l>\  the  two  independent  teams  of  ralrr> 
ranged  from  .86  to  .64  with  a  median  of  .76.  Investigation  of  the  range  of  these  team  l-leain  2  correlations  further 
emphasizes  the  great  deal  of  variability  in  the  individual  task  difficulty  levels.  In  tin*  specialties  for  which  the 
interteam  correlations  were  low.  there  is  evidence  that  conflicting  information  was  gathered  from  tin*  operational 
sites  due  to  differences  in  equipment,  automation  ol  jobs,  or  mission  requirements.  The  sensitd  it\  of  the  work  area 
experts  to  these  differences  in  site*  provides  additional  credibility  to  the  data  collection  procedures. 
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Tahir  5.  Summary  of  (iorrclalions  In'tween  Train  1  and  Team  2  Ratings 


Aptitude 

Area 

Range  nf 
r 

Median 

r 

Na 

AKS 

(General/ Administrative 

M  -  .01 

.75 

55 

Mechanical 

.<>a  -  *n 

.82 

28 

birr  Ironies 

.<>:>  -  .00 

.80 

15 

‘'Number  tit  \I'S  diller  trom  la  lilt-  I  liiTatiM1  *  w  *»  team*  were  mil  iimiI  in  iMmrliiuark  ruling  id  all  Ms. 


Kvi<l<-m-i-  of  ...ii.iirr.nl  vali«lil>  of  I  lie  ilala  rolh-cled  usinj;  I  In-  ia>k  (liffimll)  h.-mhmurk  x  alo  i>  providi-d  h\ 
corrrlalioio  bWw.rii  I  In-  average  willim->|M-<iull)  raling*  from  ini-iimlM-iil  Mi)»-r\  ix.i-  ami  llic  avt-rap.- 

Ii.-m  liiiiark  rating-.  roll.-.  I.-d  from  aptilude  area  c\|).-rl>.  As  shown  in  Table  tin-  .orr.-lalioo-  h.-lw.-.-n  ib.-s,- 
variables  ranged  from  .of  lo  ,<H  with  a  median  of  .80  for  all  sjieeiallies  studied  in  the  three  aptitude  areas  (\  = 
I  I  - ).  These  relationships  offer  support  for  the  data  eolleelion  melhodolop\  and  the  henehmark  xaling  priM-edure. 
I  lie  e\  idenee  imlieates  that  the  dill  ieultv  predictions  I  rout  benchmark  ilala  represent  a  measure  of  the  difficulty  of 
a  task  which  can  be  compared  across  as  well  as  w  ithin  specialties  in  the  same  aptitude  area.  Complete  correlational 
statistics.  summarized  in  labile  .)  and  6  an*  included  in  Appendix  A. 


Table  6.  Summary  of  Correlations  between  Within-Specialtv  Ratings 
and  Benchmark  Ratings 


Phase/ Aptitude  Area 

Range 
of  r 

Median 

r 

N 

AKS 

N 

Tasks/AKS 

Development  Phase 
(General/ Administrative 

.71  -  .80 

.81 

15 

(>0 

Mechanical 

.57  -  .01 

.77 

15 

10 

Electronics 

.81  -  .05 

.88 

15 

10 

Application  Phase 
Oeneral/Adinin 

.5 1  -  .05 

.77 

55 

60 

Mechanical 

.58  -  .88 

.81 

25 

60 

Electronics 

.51  -  .80 

.81 

22 

60 

IV.  DETERMINATION  OK  JOB  1)11  I  IU  1.TY 

( .on  vc r> ion  o(  task  difl  ietilly  into  job  difficult)  was  loo  ml  to  lie  more  complicated  (bail  a  simple  a\  cragc  of  tin* 
difficulties  of  the  tasks  comprising  the  job.  A  job  can  In*  difficult  for  a  variety  of  reasons  Mich  as  number  of  tasks, 
conditions  under  which  the  tasks  arc  performed,  variets  of  tasks.  difficulty  ol  tasks.  and  the  amount  of  lime  spent 
on  tin*  various  tasks  performed.  File  variety  of  tasks  and  the  environmental  conditions  ol  performance  did  not  lend 
themselves  lo  c|uanlifiealion.  The  number  of  tasks  performed  as  a  measure  of  job  difficulty  was  found  to  In* 
somewhat  misleading,  especially  in  comparisons  of  jobs  containing  few  \er\  difficult  tasks  with  jobs  containing 
many  simple  tasks.  Job  difficulty  was  determined  lo  In*  best  estimated  as  a  function  of  tin*  difficult  of  tin*  tasks 
comprising  tin*  job  and  the  time  spent  on  those  tasks. 


Average  Task  Difficulty  Per  I  nil  Time  -  ATDPIT 

I  In*  collection  anil  analysis  ol  task  difficulty  data  have  been  described  previously,  rime-spent  data  for 
randomly  selected  job  incumbents  in  all  Air  force  specialties  has  been  routinely  collected  and  analyzed  by  the  \ir 
force  Occupational  Measurement  (.enter  lor  the  past  several  years.  I  best*  data  an*  maintained  in  a  computer  data 
bank  ami  were  made  available  for  this  study  .In  collecting  these  data,  job  i  lieu  in  bents  are  required  to  identify  those 
ta-k-  which  comprise  his  or  her  job  and  then  indicate,  using  rclatiye  lime-spent  ratings,  tin*  rrlatiyc  lime  spent  on 
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eaeh  task  performed  compared  lo  all  other  tasks  performed  (Carpenter,  Giorgia.  &  McFarland.  1(>75).  The  data 
from  these  ratings  are  analyzed  with  the  CODAP  package.  The  relative  time-spent  ratings  provided  by  die  job 
incumbents  are  summed  and  the  rating  for  each  task  performed  is  divided  by  the  sum  of  all  ratings,  thus 
computing  a  percentage  time-spent  variable.  Previous  research  has  indicated  that  the  relative  time-spent  format 
results  in  highly  reliable  self-estimates  of  the  percentage  of  time  spent  on  tin*  various  tasks  performed  in  a  worker's 
job  (Christal,  1D74). 

Job  difficulty  for  an  individual  position  wa>  estimated  by  combining  the  predicted  task  diffic  ulty  values, 
determined  from  the  previous  analyses,  with  the  |H*rcent  time-spent  estimates  to  form  a  new  computed  variable. 
AverageTask  Difficulty  Per  l  nit  Time  (ATDPl  T).  ATDPl  T  is  simply  the  cross-product  of  percentage  time-spent 
and  task  difficult)  summed  ac  ross  all  tasks  in  the  inventory  for  an  individual  job.“  ATDPTT  can  be  computed  for 
any  group  of  individuals  (e.g..  spec  ialty  members  with  1  to  18  months  of  service)  by  summing  each  individual  s 
ATDPl  T  value  and  dividing  by  the  number  of  individuals  in  the  group.  The  CODAP  package  can  be  used  to 
compute  ATDPl  T  values  for  any  specified  group.  I  sing  the  ATDPl  T  value,  the  difficulty  level  of  individual  jobs 
or  job  types  can  be  compared  to  any  other  within  (In  same  aptitude  area  based  on  the  relative  lime  spent  and 
difficulty  of  each  task. 

The  relative  ranking  of  specialties  from  eac  h  aptitude  area  on  the  ATDPTT  value  indicates  the  relative 
difficulty  level  of  specialties  within  the  Air  Force.  Figure  1  shows  a  sample  of  specialties  from  the  General/ 
Administrative  aptitude  areas  ranked  on  ATDPl  T  value  for  enlisted  personnel  with  1  to  48  months  of  military 
service  and  their  current  ASV  \B  c  utoff  scores.  A  comparison  of  the  relative  rankings  of  the  ATDPTT  values  with 
the  ordering  of  the  ASV  AB  cutoff  scores  indicates  a  degree  of  misalignment  of  aptitude  requirements.  Specifically. 
Figure  1  suggests  that  some  specialties  currently  assigned  a  high  minimum  aptitude  requirement  may,  in  fact, 
have  a  lower  level  of  diffic  ulty  than  other  specialties  assigned  a  lower  minimum  aptitude  requirement.  Other 
specialties  were  found  to  cover  a  wide  range  of  difficulty  levels  (indicated  by  the  length  of  the  horizontal  lines  in 
Figure  I),  suggesting  that  the  specialty  might  he  divided  into  several  different  jobs. 


^ ATDPl  T  value**  are  multiplied  by 


100  to  eliminate  decimal*  and.  thus,  simplify  reporting. 
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V.  DISCI  SSION 


The  methodology  developed  and  implemented  in  this  research  can  effectively  and  objectively  be  applied  to 
evaluate  the  relative  aptitude  requirements  of  Air  Force  jobs  in  a  particular  aptitude  area.  Results  have  been 
obtained  to  substantiate  both  the  reliability  and  the  validity  of  this  methodology.  The  methodology  has  been 
applied  to  Air  Force  jobs  across  four  aptitude  areas.  Tin*  actual  realignment  of  aptitude  requirements  is  a  complex 
task  which  will  be  reported  in  a  forthcoming  report:  however,  in  this  methodology  the  Air  Force  now  has  a  valuable 
tool  for  management  and  classification.  For  the  first  time,  managers  have  systematic,  empirical  data  with  which  to 
order  jobs  relative  to  each  other  based  on  the  level  of  talent  required.  Managers  now  have  tin*  means  to  determine 
empirically  the  relative  level  of  difficulty  associated  with  newlv  developed  jobs  prior  to  setting  an  aptitude  score 
minimum.  Tlu*  availability  of  the  means  by  which  these  decisions  can  he  made  has  far-reaching  implications  for 
the  Air  Force  manpower  and  personnel  community. 

The  implications  of  the  present  study  for  the  Air  Force  classification  system  arc  particularly  relevant.  The  Air 
Force  currently  classifies  a  majority  of  enlistees  at  the  Armed  Forces  Examining  and  Entrance  Stations  via  a 
Person-Job- Match  (PJM)  algorithm  (Hendrix.  Ward.  Pina,  &  Haney.  1676).  The  PJM  system  determines  which 
specialty  to  offer  each  potential  applicant.  W  ithin  this  algorithm,  there  is  a  job  difficulty-aptitude  interaction  term 
which  increases  the  likelihood  of  an  offer  of  a  specialty  when  there  is  high  similarity  between  job  difficulty  level 
and  airman  aptitude  level.  In  other  words,  the  system  will  offer  the  most  difficult  jobs  to  the  most  talented 
applicants.  The  algorithm  is  sensitive  to  small  differences.  For  example,  at  the  time  of  this  researc  h  there  were  over 
.‘■50  Electronics  AFSs  requiring  a  minimum  composite  score  at  the  80th  cenlile.  In  this  instance,  the  PJM  algorithm 
would  likely  offer  these  jobs  more  evenly  to  all  airmen  scoring  at  or  above  E-8O011  the  Electronics  composite  of  the 
ASVAB.  However,  should  the  ATDPl  Ts  from  this  research  be  used  in  place  of  the  ASVAB  cutoff  score  in  the  job 
difficulty  component  of  the  interaction  term,  the  system  would  likely  offer  the  more  difficult  jobs  to  those  airmen 
scoring  near  65  and  the  least  difficult  of  these  E-80  jobs  to  those  scoring  near  80.  thus  providing  a  more  effective 
distribution  of  available  talent  across  jobs.  Such  a  system  would  not  override  c  urrent  ASVAB  miniinums.  but  it 
would  make  more  efficient  distribution  of  available  talent  at  or  above  the  minimum. 

The  implementation  of  these  data  into  the  PJM  algorithm  could  actually  result  in  performing  the  same 
function  as  a  change  of  aptitude  minimums.  For  example,  it  is  likely  that  some  AFSs  with  current  aptitude 
miniinums  of  40  are  nearly  as  difficult  as  other  AFSs  having  minimums  at  the  60th  eentile.  Without  changing  the 
minimums.  an  augmented  PJM  algorithm  would  tend  to  offer  the  more  demanding  job  to  individuals  having  a 
higher  level  of  talent. 

The  data  from  this  project  also  provide  Air  Force  planners  with  valuable  information  for  the  development  of 
contingency  plans  for  manning  the  force  in  the  face  of  talent  and  manpower  shortages.  Since  the  abolishment  of 
the  draft,  it  has  become  increasingly  difficult  for  the  Air  Force  to  meet  personnel  procurement  objectives.  One  of 
the  few  remaining  alternatives  for  maintaining  the  force  level  may  be  to  reduce  aptitude  levels  for  some  jobs.  It  is 
important  to  determine  how  this  might  he  accomplished  so  as  to  have  the  smallest  impact  on  mission  capabilities. 
There  are  at  least  three  ways  job  and  task  difficulty  information  could  be  used  in  preparing  such  contingency 
plans:  (a)  determine  where  aptitude  requirement  levels  could  be  reduced  for  existing  specialties,  (b)  identify 
existing  job  types  within  AFSs  which  could  be  formed  into  new  management  categories  and  manned  by  individuals 
with  less  talent,  and  (c)  identify  low-demand  tasks  in  existing  jobs  that  could  be  formed  into  new  jobs  to  be 
performed  by  individuals  with  less  talent  (Christal.  1674). 

Research  in  this  area  is  continuing.  Currently  planned  efforts  include  a  preliminary  study  of  the  extent  to 
which  the  three  benchmark  scales  overlap  and  studies  of  the  impact  that  changes  in  the  aptitude  entry 
requirements  would  have  on  the  personnel  acquisition  and  training  systems.  It  is  anticipated  that  significant 
changes  in  aptitude  entry  requirements  will  be  required.  It  is  further  anticipated  that  these  changes,  when 
implemented,  will  have  profound  effects  on  the  numbers  of  recruits  eligible  for  different  career  fields,  which  in 
turn  will  have  significant  impact  on  the  training  system.  These  studies  are  designed  to  further  explore  and  refine 
the  technology  developed  in  this  effort. 


VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


It  is  concluded  that  the  methodology  for  using  job  difficulty  indices  and  time-spent  data  as  the  basi*  for 
tie  lerinining  the  relative  aptitude  level  of  an  Air  Force  job  is  technically  feasible.  This  methodology  also  provides  a 
workable  system  lor  altering  aptitude  niiniiiiiiins  in  the  face  of  fluctuations  in  the  availability  of  manpower 
resources  with  the  least  impact  on  mission  capabilities.  Since  the  utility  of  this  methodology,  when  used  in  the 
initial  classification  process,  would  insure  a  more  effective  distribution  of  available  talent  across  jobs,  it  is 
recommended  that  this  methodology  be  considered  lor  use  in  operational  realignment  of  aptitude  requirements. 
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Table  Al«  Specialties  Used  in  the  Development  and  Application  of  the  Mechanical  Benchmark  Scale 
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APPEftDIX  B:  MECHANICAL.  GENERAL.  AND  ELECTRONICS 
BENCHMARK  SCALES 
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MECHANICAL  BENCHMARK  SCALE 


LEVEL  1 

POLICE  GROUNDS  FOR  UTTER 
POLICE  OPEN  STORAGE  AREAS 

LEVEL  2 
CUT  WEEDS 
DISPOSE  OF  RAGS 

LEVEL  3 

LUBRICATE  CABLES 
RAKE  BAR  SCREENS 

LEVEL  4 

LUBRICATE  HAND  TOOLS 

STENCIL  DATE  OF  INSPECTION  ON  LIFE  RAFTS 

LEVEL  5 

CLEAN  LIFE  PRESERVERS 
DIG  DITCHES  BY  HAND 

LEVEL  6 

CLEAN  PAINT  EQUIPMENT  SUCH  AS  BRUSHES  OR 
ROLLERS 

APPLY  REFLECTIVE  TAPE  TO  EQUIPMENT 
LEVEL  7 

REMOVE  OR  REPLACE  VENETIAN  BLINDS 
CLEAN  EQUIPMENT  OR  AREAS  AFTER  APPLYING 
PROTECTIVE  COATINGS 

LEVEL  8 

MAINTAIN  TOOL  CRIBS 
MIX  CONCRETE  BY  HAND 

LEVEL  9 

POSITION  NONPOWERED  GROUND  EQUIPMENT 
AROUND  AIRCRAFT 

APPLY  ENAMELS  TO  SURFACES  USING  ROLLERS 
LEVEL  10 

CLEAN  AND  REGRAP  SPARK  PLUGS 

CAULK  AREAS  AROUND  WINDOWS,  SINK,  OR  BATHTUBS 

LEVEL  II 

PERFORM  OPERATOR  INSPECTIONS  OR  MAINTENANCE 
ON  DUMP  TRUCKS 
DRAIN  ENGINE  OIL  SYSTEMS 

LEVEL  12 

REMOVE  OR  REPLACE  NOZZLES  OR  HOSES  ON  MOTOR 
CASUNE  UNITS 

PREPARE  ENAMELS  FOR  APPLICATION 
LEVEL  13 

INSTALL  OR  REPLACE  WATER  FOUNTAINS 
DISASSEMBLE  OR  CLEAN  CONVENTIONAL  FUEL  GATE 
VALVES 


LEVEL  14 

PRIME  COMPONENTS  SUCH  AS  STARTERS  AND 
HYDRAULIC  PUMPS 

DISASSEMBLE  OH  CLEAN  CONVENTIONAL  FUEL 
LUBRICATED  PLUG  V  ALVES 

LEVEL  15 

PERFORM  PREOPERATION AL  INSPECTIONS  OF  ENGINE 
AFTER  ENGINE  HAS  BEEN  ON  LONG  STANDBY 
INSTALL  OR  REPLACE  FORMICA  ON  COUNTER  TOPS  OR 
SPLASH  BOARDS 

LEVEL  16 

REMOVE  OR  INSTALL  CANOPY  HOSES  OR  TUBING 
PRIME  AND  BLEED  FUEL  SYSTEMS 

LEVEL  17 

REMOVE  OR  REPLACE  TRANS  MISSION- DRIVEN 
GENERATORS 

ADJUST  AUTOMATIC  GOVERNORS  AND  VOLTAGE 
REGULATORS 

LEVEL  18 

TROUBLESHOOT  HIGH  OR  LUBE  OIL  PRESSURE 
INSTALL  Ft  EL  MANIFOLDS  AND  FUEL  NOZZLES 

LEVEL  19 

INSTALL  ELECTRICAL  COMPONENTS 
REMOVE  OR  INSTALL  FUEL  CELLS 

LEVEL  20 

READ  AND  INTERPRET  SCHEMATIC  OR  WIRING 
DIAGRAMS 

INSTALL  TAIL  ROTOR  ASSEMBLIES  ON  HELICOPTER 
AIRCRAFT 

LEVEL  21 

REMOVE  OR  INSTALL  TAIL  DRIVE  ASSEMBLY 
DIRECT  AIRCRAFT  EXPLOSIVE  HAZARD  RENDER  SAFE 
PROCEDURES 

LEVEL  22 

PERFORM  CRITICAL  MEASUREMENTS  ON  JET  ENGINES 
ADJUST  CANOPIES 

LEVEL  23 

REMOVE  OR  REPLACE  CYCLIC  CONTROL  SYSTEM 
COMPONENTS 

REMOVE  OR  INSTALL  MAIN  ROTOR  TRANSMISSION 
LEVEL  24 

TROUBLESHOOT  FULLY  ARTICULATED  ROTOR  S\  SEMS 
AND  DETERMINE  CORRECTIVE  ACTIONS 
ASSEMBLE  MAIN  ENGINE  SECTIONS 

LEVEL  25 

TROUBLESHOOT  SYSTEMS  FOR  BREAKER  TRIP -OUTS 
TROUBLESHOOT  INSTALLED  ENGINES  _ 


GENERAL/ ADMINISTRATIVE  BENCHMARK  SCALE 


LEVEL  I 

Cl FAN  OR  VI  ANII  MILITARY  VEHICLES 

st  \m:  pi  blications  ok  reports 

LEVEL 2 

PREPARE  IDENTIFICATION  BANDS  FOR  PATIENTS 
OPERATE  FACILITY  LOCKS  OR  DOORS 

LEVKL 3 

COLLATE  p|  BLICATIONS 
APPLY  COLD  COMPRESSES 

LEVEL  t 

PREPARE  NEWSPAPER  OR  OTHER  PRINTED 
MATERIALS  FOR  MAILING 
STENCIL  IDENTIFICATION  NVMBERS  ON  LIFE 

svpport  eqvipment 

LEVEL  5 

TAKE  OR  RECORD  TEMPER  ATI ’RES 
SEC l  RE  WEAPONS  IN  WEAPONS  STORAGE 
LOCKER 

LEVEL  6 

INSTALL  OR  REMOVE  SI NCLE- VISOR 
ASSEMBLIES  ON  HELMETS 
ADVISE  INDIA  IDVALS  OF  THEIR  RIGHTS 
LNDER  THE  FIFTH  AMENDMENT 

LEVEL  7 

PREPARE  REINVESTS  FOR  MEDICAL/DENTAL 
RECORDS  OR  INFORMATION  FORMS 
PROVIDE  GUDANCE  TO  INSTALLATION 
VISITORS 

LEVEL  8 

PREPARE  OR  MAIL  MEDICAL  INFORMATION  TO 
REINVESTING  AGENCIES 

CHALLENGE  OR  IDENTIFY  I  NK  NOW  N  PERSONS 
LEVEL  9 

CONDI  CT  TOVRS  THROVC.H  FACILITIES 
TAKE  OR  RECORD  RADIAL  PVLSE 

LEVEL  10 

INSPECT  SITE  OR  FACILITIES  FOR  SLIPPING 
HAZARDS 

GVARD  CLASSIFIED  MATERIAL  AT  ACCIDENT 
OR  INCIDENT  SCENES 


LEVEL  I  % 

BRIEF  PERSONNEL  PRIOR  TO  APPEARANCE  ON 
RADIO  OR  TV 

assemble  si  h\  in  ai  kits  i  oh  specific 

MISSIONS 


LEVEL  15 

MAINTAIN  RASE  MASTER  PLANS 

COM  PI  TE  AIRCREW  \\  All  ABILITY 

LEVEL  10 

EST ABA  I.ISH  LOC  ATION  OF  EXISTING 
TOpin.K  APUIcvi  I  V ATI  RFs 

MAM  ALIA  PROCESS  COLOR  REVERSAL  FILM 

LEVEL  17 

CONDVCT  INTERVIEWS  IN  CONNECTION  WITH 
STORY  ASSIGNMENTS 

REVIEW  SOVRCE  MATERIAL  TO  DETERMINE 
PORTIONS  I  S  ABLE  FOR  PROJECTS 

LEVEL  18 

DETERMINE  REBUKED  GR  ADES  AND  MR  FORCE 
SPECIALTY  CODES 

W  RITE  RADIO  SCRIPTS 

LEVEL  19 

DIRECT  OPER  ATION  OF  AKROMEDlCAl.  E\  ACW 
TION  FACILITIES 

ANALYZE  TENANT  WORKLOAD  DATA  TO 
DETERMINE  HOST  MANPOWER  IMPACT 

LEVEL  20 

GHOST  WRITE  EDITORIALS 

CONDI  CT  REHEARSALS  OF  TV  PROGRAMS 

LEVEL  21 

DETERMINE  BENEFITS  DERIVED  FROM  EACH 
ALTERN  ATIVE  METHOD  OF  ACCOMPLISHING 
OBJECTIVES 

EVALl  ATE  OR  V  ALIDATE  NEED  FOR  IMMVIDV  VL 
MANPOWER  A  FOMENTATION  POSITIONS 

LEVEL  22 

BVILD  VP  LIFE  SVPPORT  MOBILITY 
CONTAINERS 

ADMINISTER  PRIMARY  CARE  AT  SCENE  OF 
ACCIDENTS 


LEVEL  1 1 

T  AKE  OR  RECORD  APICAL  PVLSE 
INSPECT  OXYGEN  MASKS  OR  ACCESSORIES 

LEVEL  12 

COORDINATE  COMPLETION  OF  CLINICAL 
RECORDS  WITH  PHYSICIANS  OR  NVRSING 
STAFFS 

CONTROL  ENTRY  AND  TRAFFIC  AT  DISASTER 
SCENES 


LEVEL  23 

DIRECT  R  ADIO  OR  TELEVISION  PROGRAMS 
PERFORM  TRIAGE  DI  KING  DISASTER 
SITIATIONS 


LEVEL  24 

WRITE  STAFF  ST l  DIES.  SVRVKYS  OR  SPECIAL 
REPORTS 

ADVISE  PERSONNEL  ON  CIVILIAN  HEALTH  AND 
MEDICAL  PROGRAM  OF  THE  VMFORMEU 
SERVICES 


LEVEL  13 

INSTRVCT  STl  DENTS  IN  METHODS  OF 
PROTECTING  FOODS  FROM  ENVIRONMENT 
OR  ANIMAO 

SELECT  FILTERS  FOR  PENETRATING  HAZE 


LEVEL  25 

PREPARE  M  ANAGEMENT  ADVISORY  REPORT 
DESIGN  INTERIOR  VTIL1TIES  SYSTEMS 
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ELECTRONICS  BENCHMARK  SCALE 


LEVEL  I 

REMOVE  AM)  DISPOSE  OF  TRASH.  WASTE  OR 
M  A  TER  LAGS 

CLEAN  OR  M  AINTAIN  AREAS  Ol  TSIDE  OF  SHOP 
LEVEL  2 

CLEAN  OR  VACI'LM  KQl  IPMKNT 

POLISH  OR  W  AX  Eg 1 1  PM  ENT  OR  F  ACILITIES 

LEVEL  3 

CLEAN  AND  MAINTAIN  HAND  TOOLS  OR  TOOL 
BOXES 

MONITOR  CLOSED  CIRCU  IT  TELEVISION 
LEVEL  4 

INSPECT  AND  CLEAN  POLL  WEATHER  GEAR 
INFLATE  OR  DEFLATE  VEHICLE  TIRES 

LEVEL  5 

STENCIL.  DECAL.  OR  PAINT  INSTRUCTIONS 
OR  IDENTIFIERS  ON  Kyl'IPMENT 
CLEAN  OR  LI  BRICATE  MECHANICAL  DEVICES 
SCCH  AS  GEARS  OR  HINGES 

LEVEL  6 

CLEAN  AND  INSPECT  LIGHTING  FIXTl  RES 
PERFORM  TOOL  BOX  INVENTORIES 

LEVEL  7 

VLSI  ALLY  INSPECT  BATTERIES 
READ  SERVICE  METERS 

LEVEL  8 

PERFORM  VLSI  AL  INSPECTION  OF  R  A  DOMES 
CLEAN  AND  TIN  SOLDERING  EQUIPMENT 

LEVEL  9 

VLSI  ALLY  INSPECT  ELECTRICAL  BONDS 
AND  GROCNDS 

INSTALL  MOl  YriNG  BRACKETS  OR  FIXTl  RES 
LEVEL  10 

INSTALL  CRIMPED  WIRING  TERMINALS  ON 
COMPONENTS.  LINE  REPLACEABLE  t  NITS. 

OR  MODI  LE  W  IRING 

INSPECT  ELECTRICAL  OITLETS  FOR  CROC  NDING 
LEVEL  1 1 

REMOV  E  OR  INSTALL  CELLS  OR  STRAPS  ON 
NICKEL  CADMII  M  OR  SILVER  /INC 
BATTERIES 

VISE  ALLY  INSPECT  WIRE  HARNESSES.  CABLES. 
OR  CONNECTOR  PLCGS 

LEVEL  12 

SOLDER  WIRES  TO  CONNECTOR  PLCGS.  CON¬ 
TROL  BOXES.  OR  CONTROL  PANELS 
VISE  ALLY  INSPECT  PRESSl  RE  W  ARNING 
CIRCUTS 


LEVEL  13 

INSPECT  OR  OPER  ATION  ALLY  CHECK 
I1YDRAI  Lie:  PRESSl  RE  INDICATING 
SYSTEMS 

INSPECT  WINDSPEED  TR  ANSMITTING  OR 
MONITORING  EgCIPMENT 

LEVEL  14 

ADJIST  TR  ANSMISSOMETER  PROJECTOR  LAMP 
VOLTAGES 

REPL  ACE  MECHANICAL  COMPONENTS  SCCH  AS 
BE  ARINGS.  GEARS.  OR  PELLETS 


LEVEL  15 

INSPECT  OR  OPERATIONALLY  CHEC  K  SI  RE  ACL 
OR  FLAP  POSITION  INDICATING  SYSTEMS 

REMOVE  OR  REPL  AC  E  SOCKETS  FOR  COMPONENTS 
SCCH  AS  Tl  BKS.  RELAYS.  AMP.  TRAN¬ 
SISTORS  OR  INTEGRATED  CIRC l  ITS 

LEVEL  16 

ADJI  ST  AMPLIFIER  BALANCES 

PERFORM  POWER  CHECKS  OF  COMMl  NICATIONS 
SYSTEMS  INSTALLED  ON  A I  BCR  A  l  r 

LEVEL  17 

ALIGN  OR  ADJI'ST  TRANSMISSOMETER  l  NITS 

PERFORM  HICH-V Al.l  E  DKSOLDKRING 

LEVEL  Itt 

ADJI  ST  OH  ALIGN  RADAR  HEIGHT  INDICATOR 
RANGE  MARK  GENERATING  CIRCUTS 

TRIM  BLESHCM)T  CONVENTIONAL.  NON -VERTICAL 
SCALE  INSTRCMENT.  KI  EL  FLOW  IN¬ 
DICATING  SYSTEMS  ON  AIRCRAFT 

LEVEL  19 

ADJI  ST  OR  ALIGN  VIDEO  AMPLIFIERS 

TROl  BLESHOOT  WIND  MEASl  RINC;  SETS 

LEVEL  20 

TROl  BLESHOOT  CONST  ANT  SPEED  DRIVE 
CIRCUTS 

ALIGN  OR  A  DJI  ST  AZIMCTH  AND  ELEVATION 
ANGLE  DETECTION  CIRCUITRY 

LEVEL  21 

TROCBLESHOOT  AIRCRAFT  FLIGHT  CONTROL 
CIRCUTS 

ADJI  ST  OR  ALIGN  ELECTRONIC  COCNTER 
COl  NTERMEASl  RES  URU  ITS 

LEVEL  22 

TROl  BLESHOOT  POWER  SI  PPLIES  AND  DIS- 
TRIBl  TIONS  ON  DIGITAL  COMPl  TERS 

PERFORM  ALIGNMENT  OF  AIRCRAFT  HF  RECEIVER 


LEVEL  23 

TROl  BLESHOOT  REGCI.ATOR  CIRCUTS  ON 
DEVICES  WHICH  CSE  AN  AN  A  LOG  COMPl  TER 
PERFORM  ALIGNMENTS  OR  A  DJI  STM  ENTS  OF  AN/ 
A  PM -335  RADAR  RECEIVER  TRANSMITTER 
TEST  SETS 

LEVEL  24 

ISOLATE  MALFUNCTIONS  IN  SYNCHRONIZER 
CIRCITTRY 

PERFORM  ALIGNMENTS  OF  ADJl  STM  ENTS  OF  AN/ 
A  PM -336  RADAR  VIDEO/SERVO  TEST  SET 


LEVEL  25 

PERFORM  FACET  ISOLATION  OF  AV  APM-336 
RADAR  VIDEO  SERVO  TEST  SET 
PERFORM  ALIGNMENTS  OR  ADJl  STM  ENTS  OF 
I2A968II  PENETRATION  AID  TEST 
STATIONS 


APPENDIX  C.:  EXCERPTS  FROM  THE  PROCEDURAL  GUIDE  FOR  USE 
OF  THE  MECHANICAL  BENCHMARK  SCALE 


PART  I 


assessing  learning  difficulty  and 

RATING  TASKS 

1.0  introduction 

You  an*  a  member  of  a  pan**!  that  will  assist  On*  USAF  Human  Resources  Laboratory  by  providing  data  on  tin* 
'’learning  difficulty  "  of  selected  Air  Form  tasks.  Tin*  Guide  drsrribrs  tin*  procedure  by  which  these  task*  are  to  be 
rated.  \ ou  will: 

(1)  Learn  to  use  a  specific  research  method  in  judging  learning  difficulty. 

(2)  Oo  to  places  where  work  is  being  performed  and  study  the  tasks  in  their  workplace. 

(3)  Rate  each  selected  task  using  a  25-level  Benchmark  Rating  Scale. 

I  bis  guide  consists  of  two  parts.  Part  I  explains  bow  to  follow  the  recpiired  research  procedure  and  bow  to  use 
the  specific  definition  of  "learning  difficulty."  That  definition  is:  "time  required  to  learn  to  perform  the  task 
satisfactorily." 


2.0  PURPOSE  OF  DIFFICULTY  RATINGS 


2.1  Background 

Since  1058  the  Air  Force  Human  Resources  Laboratory  has  been  developing  a  bank  of  scientific  data, 
concerning  the  v  arious  kinds  of  work  performed  in  the  Air  Force.  As  a  result,  most  Air  Force  Specialties  (AFSs)  can 
now  be  described  by  a  list  of  several  hundred  specific  tasks  that  are  performed  by  personnel  in  that  specialty. 
These  list>  are  in  the  form  of  task  inventories,  and  they  were  derived  from  surveys  of  workers  and  supervisors. 
Each  listed  task  is  one  which  is  actually  performed  by  personnel  in  the  AFS.  as  reported  by  the  survey. 

The  task  inventories  include  data  about  each  task,  such  as  the  frequency  with  which  it  is  performed,  how 
many  people  perform  it.  and  its  relative  difficulty.  These  data  are  used  both  in  research  and  for  many  practical 
management  decisions.  Task  inventories  are  used  in  designing  training,  in  determining  career  ladders,  and  in 
setting  minimum  scores  on  the  Armed  Services  Vocational  Aptitude  Battery,  a  battery  of  tests  required  for  entry 
into  specific  career  fields. 

2.2  Difficulty  Data 

In  an  earlier  survey  NGOs  in  each  AFS  were  asked  to  provide  task  difficulty  data.  As  a  result  of  their  input,  the 
task  inventories  now  include  a  difficulty  rating  for  each  task  in  the  list.  Those  ratings  tell  only  how  difficult  each 
task  is  compared  to  other  tasks  in  the  same  AFS.  They  do  not  tell  how  tasks  in  different  AFSs  compare  with  each 
other.  For  instance,  using  those  ratings  there  is  no  way  to  compare  the  work  of  a  medical  technician  with  that  of  a 
security  policeman. 

The  procedure  this  Guide  describes  will  be  used  to  develop  difficulty  ratings,  based  on  a  common  rating  scale, 
for  Air  Force  jobs  with  mechanical  aptitude  requirements. 


3.0  THE  PANEL 


3.1  General 

The  panel  of  which  you  are  a  member  will  rate  selected  tasks  in  each  of  several  AFSs.  These  tasks  do  not  cover 
all  work  performed  in  the  AFS  concerned,  but  they  are  a  representative  sample  of  the  task  inventory  for  each  AFS. 
The  ratings  the  panel  provides  will  he  used,  following  a  statistical  method,  to  evaluate  learning  difficulty  for  all 
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la>ks  in  the  inventory.  Thus,  the  rulings  your  panel  makes  will  set  difficult)  for  all  tasks  in  each  AFS.  Ii  is  llu  rrlnn- 
very  important  that  the  rulings  you  make  be  performed  with  rare,  using  the  e\ael  definition  and  eriteria  this 
handbook  desc  ribes.  Kaelt  rat i ng  you  make,  when  averaged  with  oilier  rating!',  will  determine  tin*  difficult)  rating 
assigned  to  mail)  other  tasks.  Final  ratings  will  he  an  important  determiner  ol  how  jolts  and  people  arc*  managed  in 
tin*  Air  Force*. 

3.2  Panel  Training 

\  ou  will  he  required  to  study  the*  procedure*  and  to  undertake  at  h*ast  two  training  exercise's.  Before*  the*  panel 
assembles,  each  panel  member  must  re*ad  and  understand  this  part  of  this  Guide.  and  study  the*  benchmark  tasks  of 
Part  11  in  detail.  Vi  hen  the  panel  assembles,  there  will  be  a  practical  exercise.  The  panel  will  be  briefed  on 
procedures  and  all  cpiestions  will  be*  answered.  Panel  members  will  then  he*  given  several  gencralK  familiar  task*. 
They  will  have  lime  to  discuss  those  tasks  anti  to  ask  questions.  Then  they  will  rate  the  fain ili.tr  tasks  using  the 
benchmark  scale.  Panel  members  will  compare*  the*ir  ratings  for  each  task  to  determine  how  well  the  ratings  are  in 
agreement.  Panel  members  will  be  asked  to  explain  why  they  made  each  rating.  They  will  discuss  how  thr\ 
interpreted  difficult)  of  the  task,  and  how  they  interpreted  the  benchmark  scale,  in  order  to  clarifv  an\ 
misunderstanding  of  the  method  or  of  the  benchmark  sc  ale. 

3.3  Materials 

The  mate*rials  provided  to  you  will  include  this  Guide,  task  lists,  and  rating  she*ets.  The  Task  Fist  sheets  have- 
space  for  taking  note's.  It  will  he*  useful,  however,  to  have  a  pad  of  paper  for  any  additional  notes  that  max  he* 
required. 

4.0  RATING  PROCEDIRF 

FI  General 

Kuril  task  is  rute*d  by  (I)  understanding  how  the  task  is  performed.  (2)  analyzing  how  difficult  it  is  to  learn. 
(3)  comparing  it  to  tasks  on  the  Benchmark  Rating  Scale,  and  (4)  r wording  the  difficulty  level  of  the  most 
comparable  tasks  on  the  rating  scale. 

4.2  Task  Assessment 

It  usually  will  not  be  clear,  just  from  a  task  statement,  what  any  given  task  entails.  Therefore,  tin*  panel  w  ill  go 
to  a  typical  I  SAF  workplace  to  study  how  each  task  is  performed  and  what  must  be  learned  to  perform  it.  Ideallv. 
we  would  like  to  observe  the  actual  performance  of  each  task.  This  is  randy  practical  and  would  re*quire  repeated 
observations  of  eac  h  task  to  be  meaningful.  Therefore,  the  principal  method  of  study  will  he*  to  interview  worker*. 
The  panel  will  visit  workers  in  their  actual  workplace  in  order  to  examine  the  equipment,  tools,  regulations,  task 
orders,  and  other  conditions  of  the  job. 

The  te*am  should  interview  at  least  two  holders  of  the  AFS  studied.  During  the  interview  panelists  should  take 
notes,  hut  they  should  not  rate  the  tasks  until  later.  Do  not  hurry.  Be  sure  all  mein  hen*  of  the*  panel  fully 
understand  each  task  before  proceeding  to  the  next  one.  Interviews  must  be  held  in  a  group,  with  all  workers  and 
panel  members  participating. 

4.3  Task  Assessment  Criteria 

Workers  should  he  interviewed  to  determine  exactly  what  each  task  is.  how  it  is  performed,  and  w  hat  skills  or 
knowledge  are  required  to  perform  it  adequately.  Study  the  following: 

(1)  Task  Definition:  W  hat  is  the  task?  First,  elear  up  any  confusion  about  what  the  task  statement  means.  W  e 
generally  know  what  a  task  is  when  we  know  what  materials  the  worker  begins  with  and  what  the  task  end-product 
is  like.  What  are  the  boundaries  of  the  task?  Find  out  what  is  and  is  not  included  in  task  performance.  This  is  a 
common  area  of  confusion.  If  the  task  is  changing  spark  plugs,  must  other  components  (air  filter,  compressor)  be 
removed  first?  Or  is  this  a  separate  task? 


(2)  The  minibrr  of  steps  in  a  task:  Tasks  that  have  many  different  steps  are  obviously  more  difficult  to  learn 
than  those  which  have  only  a  few  steps.  Tasks  that  contain  many  repetitions  of  the  same  step,  however,  may  be 
relatively  easy  to  learn. 

(3)  Tools  and  equipment  unique  io  the  task:  The  learning  time  required  for  tools  and  equipment  unique  to  a 
task  adds  to  learning  difficult). 

(1)  Regulations,  manuals  and  standard  operating  procedures:  How  detailed  is  the  documentation?  The  more 
detailed  it  is.  the  less  has  to  be  learned.  Some  tasks  do  not  have  to  be  learned,  because  they  can  be  performed  by 
simply  following  written  instructions. 

(3)  Memorization:  Does  the  task  or  any  portion  of  the  task  have  to  be  memorized  in  order  to  be  performed? 
This  adds  to  learning  difficulty. 

(<>)  Standards  of  Performance:  Tasks  differ  in  what  level  of  quality  or  real iabil ity  is  required  for 
"satisfactory  performance."  For  example,  packing  a  parachute  requires  a  higher  standard  of  product  reliability 
titan  does  changing  a  faucet  washer.  In  the  latter  case,  if  the  faucet  leaks,  you  can  do  it  again. 

(7)  Time  Criticality:  A  task  that  must  be  performed  within  a  time  limit  is  more  difficult  to  learn  than  the 
same  task  with  no  limit  for  performance. 

(8)  For  many  career  fields  there  are  required  basic  skills  or  knowledge  (typing,  mathematics).  In  some  cases 
these  are  taught  in  the  l  SAF  Technical  School.  These  skills  and  knowledges  add  to  the  learning  difficulty  of 
individual  tasks  only  to  the  extent  that  they  are  used  in  the  performance  of  that  task. 

Finally,  keep  in  mind  during  your  assessment  that  you  are  judging  "learning  difficulty  "—the  time  required 
to  learn  to  [lerform  the  job  satisfactorily.  It  includes  only  the  learning  time  unique  to  the  task  being  rated. 

1.4  Rating  the  Tasks 

After  having  studied  the  task,  each  panel  member  should  be  confident  that  he  understands  the  task,  ideally  to 
the  point  at  which  he  could  perform  it  himself.  He  must  know  the  starting  point,  the  conditions  of  performance,  the 
task  steps,  and  the  criteria  for  a  satisfactory  task  product.  He  should  have  a  set  of  notes  from  which  he  can  recall  the 
task  and  remember  what  skills  or  knowledge  are  required  in  its  performance. 

Then  each  panel  member  will  be  given  time  to  make  an  assessment  of  difficulty,  in  private,  using  the 
Benchmark  Rating  Scale. 

1.1.1  Isolate  learning  Time .  Panel  members  must  carefully  consider  each  task  and  determine  how  difficult  it  is  to 
learn.  This  means  that  they  must  recognize  the  difference  between  how  hard  the  task  is  to  perform  and  how  hard  it 
is  to  learn.  Only  learning  time  should  be  considered  as  part  of  task  difficulty.  Do  not  include  learning  time 
associated  with  the  basic  skills  and  knowledge  personnel  should  have  for  entry  into  the  Air  Force. 

1.2.2  Task  Ratings .  Each  task  to  be  rated  must  now  be  compared  with  the  tasks  on  the  benchmark  scale.  Then,  for 
each  task  to  be  rated,  find  a  difficulty  level  on  the  benchmark  scale  which  most  closely  corresponds  to  the 
difficulty  level  of  the  task  to  be  rated.  Verify  this  selection  by  reviewing  those  tasks  on  the  benchmark  scale  which 
are  at  the  levels  above  and  below  your  selection,  ensuring  that  the  tasks  above  are  more  difficult  to  learn  and  those 
below  are  less  difficult  to  learn.  Record  your  rating. 

(1)  Remember  to  consider  each  task  in  terms  of  learning  difficulty  —  not  how  hard  it  is  to  perform. 

(2)  If  one  of  the  tasks  at  a  level  appears  not  to  be  helpful,  consider  only  the  other  task  at  that  difficulty  level. 

(3)  If  you  disagree  with  the  rating  of  both  tasks  at  any  level,  use  tasks  above  and  below  that  level  for 
comparison. 
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4.4.3  Reassess.  Especially  during  yuur  first  few  days  using  this  procedure,  you  will  make  judgments  that  you  will 
want  to  reconsider  later.  This  is  because  you  are  in  the  process  of  learning  how  to  use  the  procedure,  and  because  it 
takes  time  to  become  familiar  with  the  rating  scale.  Most  important  of  all,  you  will  learn  a  great  deal  about  how  to 
observe  and  analyze  work. 

Therefore,  panel  members  are  encouraged  to  reassess  their  /atings  periodically,  and  are  required  to  rerate 
those  tasks  about  which  they  form  a  new  opinion. 
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PART  II 


DETAILED  DESCRIPTIONS  OF  BENCHMARK  SCALE  TASKS3 


M  1-1 

Level  I:  POLICE  GROUNDS  FOR  LITTER  (Construction  Equipment  Operator  -  AFSC  55151) 

Task  Performance:  This  task  is  the  routine  policing  of  grounds  around  a  heavy  equipment  compound  or  around 
troop  quarters.  On  direction  of  a  supervisor,  litter  is  picked  up  by  hand,  and  disposed  of  in  cans  or  in  a  dumpster. 

Skill/Knowledge  Required :  No  skill  or  knowledge  is  required  which  must  be  learned  in  the  service.  The  work  is 
performed  using  basic  skills,  learned  early  in  life  by  everyone. 

M  9-1 

Level  9:  POSITION  NON-POWERED  CROLLND  EQUIPMENT  AROUND  AIRCRAFT  (Aircrew  Egress  Systems 
Repairman  -  AFSC  42352) 

Equipment:  Non-powered  ground  equipment  includes  work  stands,  hoists,  slings,  seat  dollies,  canopy  stands  or 
dollies,  and  fire  extinguishers.  Most  have  wheel  locks.  Most  work  stands  have  hand  rails  which  are  installed  at  the 
time  they  are  positioned.  Some  stands  include  hydraulic  lifts  for  raising  and  lowering  the  stand. 

Task  Performance:  Positioning  of  this  equipment  is  typically  part  of  another  task.  T.O.  procedures  normally  spell 
out  the  location  of  this  equipment.  Safety  is  a  big  factor  to  avoid  danger  to  personnel  and  damage  to  the  aircraft 
during  positioning.  Positioning  includes  a  visual  inspection  of  the  area  prior  to  use.  locating  the  equipment, 
moving  it  in,  locking  wheels,  installing  hand  rails,  and  operating  hydraulic  hand  pumps. 

Skill/Knowledge  Required:  The  repairman  must  have  knowledge  of  each  of  the  types  of  equipment  used  and  of 
how  they  are  positioned  around  aircraft.  Operation  of  the  equipment  is  reasonably  simple.  The  most  important 
factors  are  the  safety  of  personnel,  and  the  prevention  of  damage  to  aircraft. 


M  18-2 

Level  18:  INSTALL  FUEL  MANIFOLDS  AND  FUEL  NOZZLES  (Jet  Engine  Mechanic  -  AFSC  42052) 

Equipment:  Fuel  manifolds  and  nozzles  are  used  to  distribute  and  inject  fuel  into  jet  engines.  (1)  An  external 
manifold  consists  of  sections  of  flexible  and  solid  tubing,  which  are  connected  to  form  a  complex  yoke  around  the 
engine.  It  conducts  fuel  to  10  or  12  nozzles,  which  are  screwed  into  the  body  of  the  engine  and  which  inject  the  fuel 
into  the  combustion  chambers.  (2)  An  internal  manifold  consists  of  a  soft  metal  circular  tube,  with  8  nozzle 
clusters,  each  containing  6  screw-in  nozzles  made  of  similar  soft  metal.  The  manifold  is  mounted  around  the 
interior  of  the  engine  on  brackets  and  support  clamps. 

Both  types  of  manifold  are  very  sensitive  to  physical  damage,  being  either  fragile  or  easily  dented  and 
deformed.  They  are  installed  by  bolts  which  must  be  either  tab-locked  or  safety  wired  in  position. 

Task  Performance:  The  task  is  performed  with  the  engine  removed  and  placed  on  a  stand,  (l)  External  manifolds 
are  removed  by  disconnecting  the  nozzles  and  removing  manifold  bracket  bolts,  after  which  the  circular  manifold 
is  either  (a)  carefully  slid  off  the  end  of  the  engine,  or  (b)  removed  by  disconnecting  the  fittings  between  sections. 
The  nozzles  are  then  screwed  out  of  the  engine  body.  To  install  manifold  and  nozzles,  the  procedure  is  reversed, 
nozzles  and  mounting  bolts  are  torqued  and  safety  wired.  Extreme  care  must  be  taken  to  avoid  dents  or  bends  in 


"Due  to  the  length  of  this  section  in  the  Procedural  Guide,  excerpt*  only  are  provided.  When  used  in  rating  tasks,  each  benchmark  scale 
task  will  appear  on  a  separate  page  and  should  not  exceed  one  page  in  length. 
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manifold  lines.  Anti-seize  compound  is  required  on  nozzle  threads.  (2)  To  remove  internal  manifolds,  main  engine 
sections  must  be  removed  to  gain  access  to  the  combustion  chamber.  Then  a  special  jig  is  inserted  to  prevent  stress 
or  deformation  of  the  manifold  dicing  nozzle  removal.  The  48  nozzles  are  removed  and  the  manifold  is  removed 
by  removing  2  sets  each  of  mounting  bracket  bolts  and  support  bracket  bolts.  After  removal,  the  manifold  is  placed 
in  another  special  jig  to  relieve  any  stresses  on  the  tubing.  Installation  is  the  reverse  of  this  procedure.  Each  nozzle 
is  reinstalled  by  assembling  3  parts  in  the  proper  order,  using  the  special  jig. 

In  either  case,  this  task  must  be  performed  by  following  the  T.O.  procedures  to  the  letter.  These  procedures 
are  detailed,  with  illustrations. 

Ski  (//Knowledge  Required:  This  task  requires  delicate  skill,  in  order  not  to  damage  the  manifolds.  The  task  is 
critical  because  improper  installation  of  the  nozzles  can  cause  destruction  of  an  engine.  The  learning  required  to 
perform  this  task  includes  learning  the  general  engine  structure,  the  tool  and  jig  skills  and  the  installation 
procedure,  all  of  these  to  a  higher  level  of  precision  and  assurance  than  would  be  required  to  install  a  less  fragile 
assembly. 


M  25-2 

Let  e!  25.  TROUBLESHOOT  INSTALLED  ENCINES  (Jet  Engine  Mechanic  -  AFSC  42652) 

L juipment:  This  task  is  performed  on  jet  engines  installed  on  aircraft.  Troubleshooting  includes  isolation  of 
failure  within  the  engine  or  confirming  that  a  failure  is  not  in  the  engine  but  some  related  subsystem. 

Task  Performance :  Troubleshooting  typically  begins  with  a  pilot  write-up.  Interpretation  of  these  write-ups  is 
often  difficult.  The  isolation  process  depends  upon  the  failure  symptom  observed.  Oil  leaks,  which  are  the  most 
common  problems  require  that  all  oil  be  cleaned  from  the  exterior  of  the  engine,  the  engine  and  oil  systems 
exercised,  and  examining  for  the  source  of  oil  leaks.  Vibrations  are  isolated  by  attaching  vibration  sensors  at 
different  locations  around  the  engine  and  then  running  the  engine  to  look  for  abnormal  vibration  sources.  Other 
problems  such  as  fuel  leaks,  throttle  rigging,  fuel  control,  and  electrical  problems  require  coordination  with  other 
subsystem  specialties  to  isolate  the  problem  between  the  engine  and  related  systems. 

Ski  If /Knowledge  Required :  Learning  troubleshooting  is  accomplished  by  exposure  and  is  not  formalized.  It 
requires: 

(a)  A  complete  knowledge  of  engine  operation  and  its  interface  with  related  aircraft  subsystems. 

(b)  Ability  to  use  and  understand  the  readings  of  pressure  gauges,  vibration  sensors,  and  heat  gauges. 

(c)  That  the  mechanic  be  cockpit  qualified  to  enable  him  to  run  up  the  engine. 

(d)  An  ability  to  read  and  interpret  the  appropriate  Technical  Orders. 

(e)  Coordination  with  the  efforts  of  other  subsystem  specialists  to  isolate  problems  in  the  interaction  of  the 
engine  and  related  aircraft  systems. 
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